WIP: add an async dhcp client #1250

etene · 2025-01-14T16:27:03Z

Here is my async DHCP client implementation, still a work in progress

ffourcot · 2025-01-14T16:31:20Z

pyroute2/dhcp/constants/dhcp.py

+    CLIENT_SYSTEM_ARCHITECTURE_TYPE = 93  # Client System Architecture Type
+    CLIENT_NETWORK_INTERFACE_IDENTIFIER = 94  # Client Network Interface Identifier
+    CLASSLESS_STATIC_ROUTE = 121
+    DOMAIN_SEARCH = 119


this one is not sorted?

i added it because at some point it was missing when i was testing something, but this is still a draft, these are not even used at this point

svinota · 2025-01-14T19:42:25Z

Wow, c'est magnifique. It will take some time to read & digest, be patient :)

etene · 2025-01-14T20:52:35Z

Wow, c'est magnifique. It will take some time to read & digest, be patient :)

Thanks ! Take your time, it's still a draft and lots of things are going to change, i'm not yet proud with everything

svinota · 2025-01-15T08:40:43Z

pyroute2/ext/rawsocket.py

        # bring up the socket
        socket.__init__(self, AF_PACKET, SOCK_RAW, htons(ETH_P_ALL))
+        socket.setblocking(self, False)


@etene what would you say — could it be a reasonable idea to make a socket wrapper that would use composition, and proxy most of the properties, but fake .type to make it work with loop.create_datagram_endpoint() ? and then use asyncio.Protocol

I think it's an interesting idea.

I have tried at some point to make raw sockets work with asyncio.Protocol but changed my mind when i saw how a lot of code assumed TCP, UDP or Unix sockets iirc. But I hadn't thought about faking the socket type, and it might just work given how similar raw & UDP sockets are.

I'm willing to try if I find the time to. However, the python stdlib code might be too tightly coupled with UDP/TCP/unix sockets, so it might turn out to be dirty or impossible without making a PR to python itself, which is a whole other can of worms...

svinota · 2025-01-15T14:32:04Z

@etene JFYI: I plan to keep Python compatibility back to version 3.9 as for now. Will be there any issues with that — pls let me know.

etene · 2025-01-15T15:06:17Z

@etene JFYI: I plan to keep Python compatibility back to version 3.9 as for now. Will be there any issues with that — pls let me know.

@svinota I'm pretty sure i can make it work with 3.9 upwards, my initial draft was based on 3.12 out of habit and comfort but the really important asyncio parts have been there since 3.7.

svinota · 2025-01-15T16:15:01Z

The reason behind 3.9 (FIXME: I should document it) is socket.recv_fds() that is not supported in earlier versions — and which is used in the core functionality to init netns-enabled sockets.

svinota · 2025-01-18T08:18:14Z

@etene regarding the tests I would propose this thing:

save binary packets (optionally, store them as hex dumps, see [1])
use config.mock_dhcp in a way like [2][3]
having this, just mock the traffic through the socket, and assert every state transition
bonus: now code snippets in the docs can be tested as well, to keep the docs up to date automatically, like in [4][5]

[1]

pyroute2/tests/test_core/test_ipr/test_route_dump_async.py

Lines 7 to 23 in 899fc58

    
           from pyroute2.common import load_dump 
        
           test_dump_data = ''' 
        
           # 10.1.2.0/24 via 127.0.0.8 dev lo table 100 
        
           # 10.1.3.0/24 via 127.0.0.8 dev lo table 100 
        
           24:12:31:45  3c:00:00:00  18:00:22:00  dc:d0:86:67 
        
           7b:68:00:00  02:18:00:00  64:03:00:01  00:00:00:00 
        
           08:00:0f:00  64:00:00:00  08:00:01:00  0a:01:02:00 
        
           08:00:05:00  7f:00:00:08  08:00:04:00  01:00:00:00 
        
           3c:00:00:00  18:00:22:00  dc:d0:86:67  7b:68:00:00 
        
           02:18:00:00  64:03:00:01  00:00:00:00  08:00:0f:00 
        
           64:00:00:00  08:00:01:00  0a:01:03:00  08:00:05:00 
        
           7f:00:00:08  08:00:04:00  01:00:00:00 
        
           '''

[2]

pyroute2/pyroute2/netlink/rtnl/iprsocket.py

Lines 93 to 95 in 899fc58

    
           if config.mock_iproute: 
        
               use_socket = IPEngine() 
        
               netns = None

[3]

pyroute2/docs/iproute_linux.rst

Lines 3 to 11 in 19a309b

    
           .. testsetup:: * 
        
              from pyroute2 import config, IPRoute 
        
              config.mock_netlink = True 
        
              ipr = IPRoute() 
        
           .. testcleanup:: * 
        
              ipr.close()

[4] source:

pyroute2/pyroute2/iproute/linux.py

Lines 254 to 267 in 19a309b

    
                   .. testcode:: fm01 
        
                       # get all links with names starting with eth: 
        
                       # 
        
                       for link in ipr.filter_messages( 
        
                           lambda x: x.get("ifname").startswith("eth"), 
        
                           ipr.link("dump"), 
        
                       ): 
        
                           print(link.get("ifname")) 
        
                   .. testoutput:: fm01 
        
                       eth0

[5] rendered: https://pyroute2.org/pyroute2-0.8.1-109-g19a309be/iproute_linux.html#pyroute2.iproute.linux.RTNL_API.filter_messages

svinota · 2025-01-18T08:29:28Z

tests/test_linux/fixtures/dhcp_servers/udhcpd.py

+    def get_cmdline_options(self) -> tuple[str]:
+        '''All commandline options passed to udhcpd.'''
+        return (
+            'udhcpd',


use busybox udhcpd instead, on Fedora there is no separate link for that, unlike Ubuntu; but running the plugin this way as busybox udhcpd works on both Fedora and Ubuntu.

ffourcot reviewed Jan 14, 2025

View reviewed changes

etene marked this pull request as draft January 14, 2025 20:53

etene force-pushed the async-dhcp-client branch 4 times, most recently from 12c5eb9 to 14cdae0 Compare January 15, 2025 08:40

svinota reviewed Jan 15, 2025

View reviewed changes

etene added 2 commits January 15, 2025 09:44

WIP: add an async dhcp client

f1a9a74

add StrEnum and async sock recv compat for python < 3.11

353efde

etene force-pushed the async-dhcp-client branch from 14cdae0 to 353efde Compare January 15, 2025 08:45

etene added 5 commits January 15, 2025 10:04

remove typing.Self to restore compat with older pythons

3d009de

dnsmasq fixture: ensure consistent output language (LANG=C)

3f34271

dhcp client: indent lease files

e2ce512

dhcp client: split send/recv into separate tasks

14eaf41

add ETHERTYPE_IP compat for python < 3.12

2176061

etene added 2 commits January 15, 2025 15:42

typing fixes and compat

e1d2a4f

dhcp client: replace match/case with if/else

52937a2

etene added 2 commits January 16, 2025 17:51

dhcp client: use Optional everywhere for py39 compat

b6b752a

improve dnsmasq fixture & handling of timeouts in dhcp client tests

e0566b0

etene force-pushed the async-dhcp-client branch from f9376db to e0566b0 Compare January 16, 2025 16:51

etene added 5 commits January 16, 2025 18:28

dhcp client: add basic NAK support

2a45a26

dhcp client: handle expired leases on loading

ba3d62a

dhcp client: fix coroutine warnings in timer callbacks

e941c98

dhcp client: fix logic error when loading lease

091056d

dhcp client: add comments, improve waiting for state

1e195c4

etene force-pushed the async-dhcp-client branch from 31c1b42 to d6fa1e5 Compare January 17, 2025 19:06

dhcp client: add a udhcpd fixture and a test with it

417a8d9

etene force-pushed the async-dhcp-client branch from d6fa1e5 to 417a8d9 Compare January 17, 2025 19:13

svinota reviewed Jan 18, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: add an async dhcp client #1250

WIP: add an async dhcp client #1250

etene commented Jan 14, 2025

ffourcot Jan 14, 2025

etene Jan 14, 2025

svinota commented Jan 14, 2025

etene commented Jan 14, 2025

svinota Jan 15, 2025

etene Jan 15, 2025

svinota commented Jan 15, 2025

etene commented Jan 15, 2025

svinota commented Jan 15, 2025

svinota commented Jan 18, 2025 •

edited

Loading

svinota Jan 18, 2025

WIP: add an async dhcp client #1250

Are you sure you want to change the base?

WIP: add an async dhcp client #1250

Conversation

etene commented Jan 14, 2025

ffourcot Jan 14, 2025

Choose a reason for hiding this comment

etene Jan 14, 2025

Choose a reason for hiding this comment

svinota commented Jan 14, 2025

etene commented Jan 14, 2025

svinota Jan 15, 2025

Choose a reason for hiding this comment

etene Jan 15, 2025

Choose a reason for hiding this comment

svinota commented Jan 15, 2025

etene commented Jan 15, 2025

svinota commented Jan 15, 2025

svinota commented Jan 18, 2025 • edited Loading

svinota Jan 18, 2025

Choose a reason for hiding this comment

svinota commented Jan 18, 2025 •

edited

Loading